Publish/subscribe data processing systems have become very popular in recent years as a way of distributing data messages (publications). Publishers are typically not concerned with where their publications are going, and subscribers are typically not interested in where the messages they receive have come from. Instead, a message broker typically assures the integrity of the message source, and manages the distribution of the message according to the valid subscriptions registered in the broker.
Publishers and subscribers may also interact with a network of brokers, each one of which propagates subscriptions and forwards publications to other brokers within the network.
FIG. 1 illustrates a typical publish/subscribe data processing system according to the prior art. A message broker 15 has an input mechanism 20 which may be, for example, an input queue or a synchronous input node by which messages are input when they are sent by a publisher 5; 10 to the message broker. A published message is fetched from the input mechanism by a controller 40 and processed to determine, amongst other things, to which subscribers 60; 65; 70 the message should be sent.
Message topics typically provide the key to the delivery of messages (publications) between publishers and subscribers. The broker attempts to match a topic string on a published message with a list of clients who have subscribed to receive publications including that topic string. A matching engine 30 is provided in the message broker for this very purpose. When the subscriber registers, it must typically specify a means by which it wants to receive messages (which may be a queue or other input mechanism) and a definition of the types of messages that it is interested in. A subscriber can specify that it wishes to receive messages including a topic string such as “employee/salary” and any messages matching that topic string will be identified and forwarded on to the subscriber via an output mechanism 50. (Note, there may be more than one input and output mechanism to and from which messages are received and sent by the message broker.)
A broker typically deletes a publication when it has delivered that publication to all the interested (registered) subscribers. This type of processing is suitable for event information (e.g. a stock trade or a goal scored), but is not always suitable for a subscriber that registers subsequently and wishes to be informed of the latest state information (e.g. the current price of a stock). A broker can therefore take it upon itself to keep, for example, a copy of the last publication published on each topic. Each such copy is called a retained publication.
Such a copy can be sent to subsequent subscribers who register an interest in the topic relating to the retained publication. This means that new subscribers don't have to wait for information to be published again before they receive it. For example, a subscriber registering a subscription to a stock price would receive the latest price straightaway, without waiting for the stock price to change (by being re-published).
Thus, the last publication on each topic in a retained publication system is typically retained by the message broker (publishing broker/node) to which those publications are published. This method requires that all brokers manage (retained) publications received locally. A subscriber would then receive the retained publication that is held by the subscriber's local broker. A problem with this is that one subscriber may receive a retained publication having one contents (e.g., IBM stock price is $4) from its local broker and another subscriber may, at the same time, receive a retained publication having another contents (e.g., IBM stock price is $5) from its local broker, so there is no definitive answer as to, at a certain point of time, what is the most recent retained publication in the network.
This latter problem may be solved by having only one broker which is considered to hold the definitive version of the retained publication. However, that solution suffers from the disadvantage that there is now a bottleneck in that one broker, which can go off-line, or otherwise be unavailable.